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10)E] The drawing(s) filed on 18 August 2003 is/are: a)IE accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
11 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)13 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)IEI All b)D Some * c)D None of: 

1 Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) £3 Notice of References Cited (PTO-892) 

2) O Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) [3 Information Disclosure Statement(s) (PTO/SB/08) 

Paper No(s)/Mail Date 5/3/04 . 

U.S. Patent and Trademark Office 

PTOL-326 (Rev. 08-06) Office Action Summary Part of Paper No./Mail Date 20061 229 



4) D Interview Summary (PTO-413) 

Paper No(s)/Mail Date. . 

5) CD Notice of Informal Patent Application 

6) □ Other: . 



I ..I 

4 

\ 

Application/Control Number: 10/642,195 Page 2 

Art Unit: 2624 

DETAILED ACTION 
Priority 

1 . Receipt is acknowledged of papers submitted under 35 U.S.C. 1 1 9(a)-(d), which 
papers have been placed of record in the file. 

Information Disclosure Statement 

2. The information disclosure statement (IDS) submitted on 5/3/04, the information 
disclosure statement is being considered by the examiner. 

Drawings 

3. The Examiner has approved drawings filed on 8/18/03. 

Claim Objections 

4. Claim 2 recites the limitation wherein the input handwriting signal in step should 
be (C) and not (D). There is insufficient antecedent basis for this limitation in the claim. 

Claim Rejections -35 USC §102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 
122(b), by another filed in the United States before the invention by the applicant for patent or 
(2) a patent granted on an application for patent by another filed in the United States before the 
invention by the applicant for patent, except that an international application filed under the 
treaty defined in section 351(a) shall have the effects for purposes of this subsection of an 
application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 
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Claims 1-2, 4-12,14- 20 are rejected under 35 U.S.C. 102(e) as being anticipated 
by Bellegarda et al., (US. 6,285,785 B1, Listed in IDS filed on 5/3/04). 

As to claim 1 , Bellegarda discloses a method of integrating vocal input 
recognition and handwriting input recognition (abstract, column 5, lines 35- 40) 
comprising the steps of: 

(A) receiving a syllabic (note, syllabic corresponds to speech or voice) vocal input 
signal of a object alphanumeric symbol (fig 1 , element 22, column 5, lines 35-40, note, 
a user interface having a first input coupled to an output of a speech transducer and a 
second input coupled to an output of a handwriting transducer for receiving signals 
therefrom and for converting the signals to a first multi-dimensional representation of a 
speech signal and to a second multi-dimensional representation of a handwriting signal 
. A first transducer for converting a user's speech to a first signal and a second 
transducer for converting the user's handwriting or a second signals); 

(B) recognizing the input vocal signal and generating an alphanumeric symbol 
array having a plurality of candidate alphanumeric symbols corresponding to the object 
alphanumeric symbol (column 5, lines 14- 24, 34- 40); 

(C) receiving an input handwriting signal describing the feature of the object 
alphanumeric symbol (fig 1 , element 20, column 5, lines 35- 40); 

(D) extracting the most coincidental candidate alphanumeric symbol from the 
alphanumeric symbol array corresponding to the feature (column 6, lines 45- 63, 
column 5, lines 14- 24, and 34- 40). 
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As to claim 2, Bellegarda discloses the method of claim 1 wherein the input 
handwriting signal in step (D) is a substructure of the object alphanumeric symbol (fig 
3). 

As to claims 4, Bellegarda discloses the method of claim 1 , between step (B) and 
step (C) further comprising step (E) listing the most frequently utilized candidate 
alphanumeric symbols of the alphanumeric symbol array (column 6, lines 1-44). 

As to claim 5, Bellegarda discloses the method of claim 4, after step (D) further 
comprising step (F) listing the most coincidental candidate alphanumeric symbol of the 
alphanumeric symbol array to replace the most frequently utilized candidate 
alphanumeric symbols (column 6, lines 1-44). 

Regarding claim 6, argument analogous those presented for claim 1 are 
applicable to claim 6, as discloses by Bellegarda detecting if there exists a second 
input describing the object alphanumeric symbol during a predetermined time span; If 
there exists the second input, then extracts the corresponding candidate alphanumeric 
symbol from the alphanumeric symbol array coinciding with the second input, wherein 
the first input is whether a vocal input or a handwriting input, the second input is 
whether the vocal input or the handwriting input (column 1 1 , lines 65-67, column 12, 
lines 1-4 and 10-19). 

As to claim 7, Bellegarda discloses the method of claim 6 further comprising a 
step receiving the first input and then converting the first input into a first signal 
(abstract, note, first signal corresponds to speech, fig 1 , element 22); 
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a step receiving the second input and then converting the second input into a 
second signal (abstract, note, second signal corresponds to handwriting, fig 1 , element 
20). 

As to claim 8, Bellegarda discloses the method of claim 6 further comprising a 
step extracting the feature of the first input from the first signal; and 

a step extracting the feature of the second input from the second signal (note, the 
first and second signal are processed to decode a consistent message conveyed 
separately by the first and second signal (fig 1, 44, column 3, lines 63- 67, column 4, 
lines 1 - 3), speech sample being processed to obtain a feature vector by the 
determination of amplitudes associated with a plurality of frequency bands, (fig 4c, 
column 6, lines 46-50, column 11, lines 1 1-19), handwritten character being processed - 
to generate a feature vector based on the electronic tablet for the handwriting in 
which transducer typically acquired points by collecting a plurality of spatial attributes 
per second (fig 4b). The sampled handwriting is applied to an associated handwriting 
feature vector is a set of characteristics of the input accumulated during a specific time 
interval. Processor generates one or more handwriting feature vector for each input 
stroke (column 6, lines 26- 45), collecting a plurality of spatial attributes (fig 4b). 

As to claim 9, Bellegarda discloses the method of claim 6 further comprises a 
step extracting the most frequently utilized candidate alphanumeric symbol from the 
alphanumeric symbol array where the second input does not exist (column 6, lines 46- 
50, column 11, lines 11-19). 
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As to claim 10, Bellegarda discloses the method of claim 9 further comprises a 
step representing the most frequently utilized candidate alphanumeric symbol (column 
14, lines 32-40). 

As to claim 1 1 , Bellegarda discloses the method of claim 6 further comprises a 
step representing a candidate alphanumeric symbol in accordance with the 
alphanumeric symbol (column 5, lines 41- 53, column 8, lines 36- 58, column 10, lines 
25- 31, column 11, lines 65- 67, column 12, lines 1-5). 

Regarding claim 12, argument analogous those presented for claims 6 and 8 are 
applicable to claim 12, as discloses by Bellegarda (a vocal similarity estimator for 
generating an alphanumeric symbol array having a plurality of candidate alphanumeric 
symbols corresponding to the object alphanumeric symbol according to the first signal; 
a handwriting similarity estimator for extracting the most coincidental candidate 
alphanumeric symbol from the alphanumeric symbol array according to the second 
signal, fig 2, column 12, lines 5-35, lines 58-68). 

As to claim 14, Bellegarda discloses the recognition system of claim 12 further 
comprising a vocal database storing a plurality of vocal patterns, the vocal patterns are 
provided for the vocal similarity estimator to map with the first signal and generating the 
alphanumeric symbol array (column 7, lines 6- 29, column 8, lines 36- 58). 

As to claim 15, Bellegarda discloses the recognition system of claim 12 further 
comprises a vocal feature extractor for extracting the characteristic of the vocal input 
from the first signal and transmitting the characteristic to the vocal similarity estimator 
(column 7, lines 6- 29, column 8, lines 36- 58). 



Application/Control Number: 10/642,195 Page 7 

Art Unit: 2624 

As to claim 16, Bellegarda discloses the recognition system of claim 12 further 
comprising a handwriting database storing a plurality of handwriting patterns, the 
handwriting patterns are provided for the handwriting similarity estimator to map with 
the second signal and extract the candidate alphanumeric symbol most coinciding with 
the object alphanumeric symbol (column 8, lines 13-58) 

Regarding claim 17, argument analogous those presented for claims 6 and 8 are 
applicable to claim 17, as discloses by Bellegarda (a first similarity estimator for 
generating an alphanumeric symbol array having a plurality of candidate 
alphanumeric symbols corresponding to the object alphanumeric symbol by the 
first signal (column 1 2, lines 6-1 1 , 24- 35); 

a second similarity estimator for extracting the candidate alphanumeric symbol 
that most coinciding with the object alphanumeric symbol from the alphanumeric 
symbol array by the second signal (column 12, lines 6-1 1 , 24- 35). 

As to claim 18, see the rejection of claim 12 above. 

As to claim 19, see the rejection of claim 17 above. 

Regarding claim 20, argument analogous those presented for claim 1 is 
applicable to claim 20, as discloses by Bellegarda a computer accessible recording 
medium comprises a plurality of programming codes (note, recording medium is part of 
the component system which includes the data input (receiving) components via 
microphone and handwriting reading device and data storage id the computer memory, 
these are all the components of the device and method described by algorithm is a 
program or steps to perform a given process by the computer, column 6, lines 46- 63). 
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Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
Claims 3 and 13, are rejected under 35 U.S.C. 103(a) as being unpatentable 

over Bellegarda et al., (US. 6,285,785 B1, Listed in IDS filed on 5/3/04), as applied to 

claims 1-2, 4-12,14- 20 above and further in view of Van Kleeck (US. 6,539,113 B1). 

Regarding claim 3, Bellegarda discloses message recognition method and 

apparatus that integrates handwriting recognition and speech recognition. Bellegarda 

is silent about wherein the substructure of the object alphanumeric symbol is the 

radical of the object alphanumeric symbol. 

Van Kleeck discloses radical definition and dictionary creation for a handwriting 

recognition system. The system comprises of: 

wherein the substructure of the object alphanumeric symbol is the radical of the 

object alphanumeric symbol (column 4, lines 1-14, column 6, lines 28- 33). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 

time of the invention to have modified Bellegarda to include wherein the substructure of 

the object alphanumeric symbol is the radical of the object alphanumeric symbol 

(column 4, lines 1-14). It would have been obvious to one of ordinary skill in the art at 
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the time of the invention to have modified Bellegarda by the teaching of Van Kleeck in 
which radical recognition systems have attained higher accuracy in recognition Kanji 
character over a systems to reduced the amount of data that must be stored when 
performing Kanji character recognition (as suggested by Van Kleeck at column 1 , lines 
55- 59). 

Other prior art cited 

7. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Reitano (US. 6,535,853 B1) discloses system and method for dyslexia detection 
by analyzing spoken and written words. 

Ditzik (US.6,41 5,256 B1) discloses integrated handwriting and speed recognition 
system. 

Oberteuffer et al., (US.6,438,523 B1) discloses processing handwritten and 
handdrawn input and speech input. 

Yamada et al., (US.5,692,097) discloses voice recognition method for 
recognizing a word in speech. 

Parthasarathy (US.6,275,61 1 B1 ) discloses handwriting recognition device, 
method and alphabet, with strokes grouped into stroke sub-structures. 

Ilan et al., (US.5,982,929) discloses pattern recognition method and system. 
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8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Sheela C Chawan whose telephone number is. 571- 
272-7446. The examiner can normally be reached on Monday - Thursday 7.30 - 6.00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Matthew Bella reached on 571-272-7778. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



Sheela Chawan 
Patent Examiner 
Group Art Unit 2624 
Jan 4, 2007 




